10/51188 

6X01 Rec'd PCT/PT'^ ^ 9 qcT 



The invention concerns a method for determining the 
spatial coordinates of an object according to the in- 
troductory part of the main claim as well as a device 
for carrying out the method. 

5 

For the contactless two-dimensional detection of sur- 
face shapes, surface geometries or coordinates of se- 
lected points, various optical principles are em- 
ployed. A common feature of all the methods here is 
10 that determination of the 3-D coordinates of a sur- 

face measuring point is possible only if there are at 
least three independent measured values for this 
point. In addition, assumptions about the geometry of 
the measuring system are included in the result. 

15 

One method is the conventional strip projection tech- 
nique, which is carried out with one or more CCD cam- 
eras and a projector (DE 41 20 115 C2, DE 41 15 445 
Al) . In devices of this kind, the grid lines or Gray 
20 code sequences are projected onto the surface to be 
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measured. A CCD camera records on each of its re- 
ceiver elements the intensity of a picture element on 
the surface. With known mathematical algorithms, 
phase measurements are calculated from the intensity - 
5 measurements. The desired object coordinates can sub- 

sequently be calculated from the phase measurements 
and the image coordinates of the measuring points in 
the image plane of the photographic system. A condi- 
tion of this is, however, knowledge of the geometry 
10 of the measuring system (orientation parameters of 

projector and camera) as well as of the imaging prop- 
erties of the projection and imaging lens. 

The number of orientation parameters to be determined 

15 can be limited considerably if only the phase meas- 

urements are used for calculation of coordinates. In 
such systems, the position of a single receiver ele- 
ment in the photographic system determines only the 
location of measurement, but is not evaluated as 

20 measuring information. By lighting the scene from 

several, but at least three directions of projection 
with grid lines or Gray code sequences, and viewing 
with one or more cameras fixed in position in rela- 
tion to the object, for example, coordinates for a 

25 known geometry of the lighting system can be calcu- 

lated. In all these systems the system parameters 
(orientation parameters) must be detected separately, 
this being done typically by so-called pre- 
calibration of the system. In this case, calibrating 

30 bodies of known geometry are measured, by means of 

which the geometry parameters of the measuring struc- 
ture are modelled (DE 195 36 297 Al) . This procedure 
is unusable whenever geometry parameters in further 
measurements cannot be kept constant, for example, 

35 due to temperature effects or as a result of mechani- 

cal stress on the system, or if, due to the complex- 



ity of the measuring task, a variable sensor arrange- 
ment is required and therefore measurement with a 
pre-fixed arrangement does not come into question. 

Photogrammetric measuring methods overcome the diffi- 
culty of a separate calibrating procedure. The image 
coordinates here serve as measuring information, that 
is, the position of the measuring points in the ma- 
trix of the photographic system. From at least two 
different camera positions, the image coordinates for 
an object point must be known. An advantage of these 
measuring methods is that one surplus measurement can 
be obtained to every measuring point, i.e. for two 
camera positions there is one measurement more than 
is necessary for calculation of the three coordinates 
of a point. In this way, with a large enough number 
of measuring points it is possible to simultaneously 
calculate coordinates, internal and external orienta- 
tion parameters of the cameras, and correction pa- 
rameters for recording. However, difficulties arise 
in locating the homologous points necessary for this, 
above all for a very large number of measuring 
points. For this purpose, in elaborate image process- 
ing procedures, textures or surface structures from 
different photographs must be related to each other 
(DE 195 36 296 Al) . For complete two-dimensional de- 
tection of an object surface, this is not possible 
with justifiable expense. Also, markings are required 
as link points for joining the partial views to- 
gether . 

In DE 196 37 682 Al is proposed a system which over- 
comes these problems. Here, a projection system illu- 
minates the scene with a series of strip images, con- 
sisting of two sequences turned through 90° to each 
other. Such strip images projected from two different 



4 



positions onto the object allow, at the same time 
viewing with a fixed-position camera, evaluation ac- 
cording to the functional model of photogrammetry . 
Drawbacks of this system concept arise above all in 
5 the case of complete measurement of complex objects. 

With the complexity of the object to be measured, 
there is also an increase in the number of views nec- 
essary. But it is not sensible to increase the number 
of cameras, because there is an item of measuring in- 

10 formation only at one object point which is both il- 

luminated from two different directions and viewed by 
the camera. Adjustment of the measuring system, i.e. 
orientation of the required cameras, is furthermore 
all the more difficult, the more views have to be set 

15 up. For complex measuring tasks, such prospective 

orientation of the sensor system is not always satis- 
factory. A drawback of known methods is, moreover, 
that the result of measurement is always available 
for an assessment only at the end of the complete 

20 measuring process. Intermediate evaluation and, based 

on this, adapted positioning of the projector and 
camera (s) are not possible here. 

From DE 100 25 741 Al is known a method for determin- 
25 ing the spatial coordinates of objects and/or their 

variation in time, in which the object is in each 
case illuminated from at least two directions with a 
series of light patterns which are recorded with a 
two -dimensional -resolution sensor arrangement , this 
30 being for picking up different views with different 

positions of the sensor arrangement. In this case, 
with a new position of the sensor arrangement, at 
least one direction of projection is selected so as 
to correspond to a direction of projection of the 
35 preceding position of the sensor arrangement. With 

these two directions of projection the phase measure- 
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ments are identical, and from them can be determined 
a linking rule between the recording points of the 
sensor arrangement for the new and the preceding po- 
sition. This system is self-calibrating, i.e. no geo- 
5 metrical or optical system variables have to be known 

or pre-calibrated before measurement. Calibration 
takes place with this known method during measure- 
ment, i.e. the calibrating camera is simultaneously 
the measuring camera. This method is not satisfactory 
10 with complex objects, for example, because shadows 

cannot be avoided. 



It is the object of the invention to provide a method 
for determining the spatial coordinates of an object, 

15 with which measurement of complex objects is possible 

without markings or textures, without locating ho- 
mologous points and without geometrical or optical 
system variables having to be known or pre- 
calibrated, wherein the number of possible directions 

2 0 of photography is not limited by the number of cam- 

eras and wherein the measuring time can be reduced. 

This object is achieved according to the invention by 
the characteristics of the main claim. 

25 

Due to the fact that the light patterns projected 
from the at least two directions onto the object are 
picked up as an image point by point not only in each 
case by an associated first recording device, but ad- 

30 ditionally by at least one second recording device, 

wherein the at least one second recording device re- 
mains stationary in relation to the object to be 
measured and is not moved like the projection device 
and the first recording device, and that from the 

35 points of the projected light patterns recorded with 

the second recording device are determined at least 
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four phase measurement values from which the parame- 
ters of the projection device for all directions of 
projection are determined, it is possible to perform 
calibration of the system in a separate manner from 
5 measuring. The object can be detected from any number 

of directions on account of the freely movable sensor 
arrangement, consisting of projection device and 
first recording device. The data necessary for cali- 
bration are gathered during measurement recording, 

10 separation being accomplished between stationary 

calibrating camera and movable measuring camera. This 
increases the flexibility of the measuring arrange- 
ment in the sense of a mobile sensor. The number of 
object views to be digitised is basically free and 

15 not limited to the number of cameras used, but can be 

greater. Due to self -calibration, the measuring sen- 
sor arrangement can be subject to mechanical changes 
during the measuring cycle without these affecting 
the result of measurement. 

20' 

As four phase measurement values are necessary for 
determining the geometrical parameters of the projec- 
tion device, only two phase measurement values for 
those of the first recording device, and only one 
25 phase measurement value for calculating the three- 

dimensional coordinates, the measuring time can be 
shortened considerably. * 

Due to the measures provided in the subsidiary 
30 claims, advantageous developments and improvements 

are possible. 



In general, complex objects can be measured with the 
method according to the invention. Locating marks 
35 fixed to the object or specially projected or special 

object properties for locating the homologous points 
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are not needed to build up a picture. Elaborate 
matching procedures are therefore completely elimi- 
nated. By contrast, free virtual marks are available 
through the pixel matrix of the stationary calibrat- 
5 ing camera (s). Object interaction is thus avoided. 

Technically very simple automatic measuring systems 
can be constructed. Nevertheless, these objects which 
are complex in position can be detected completely 
and two-dimensionally . For the change of position of 

10 the sensor arrangement, expensive precision guiding 

means or handling systems are not necessary. For the 
assignment of partial areas to the whole three- 
dimensional image, overlapping of the partial areas 
respectively detected with the sensor arrangement is 

15 not necessary. 

The invention is shown in the drawings and described 
in more detail in the description below. The drawings 
show : 

20 

Fig. 1 a schematic view of the measuring cycle ac- 
cording to the method of the invention. 



Fig. 2 a first embodiment of a measuring device 
25 used with the method of the invention. 

Fig. 3 a second embodiment of a device for carrying 
out the method of the invention, 

30 Fig. 4 a third embodiment of a measuring device for 

carrying out the method of the invention. 



35 



Fig. 5 



a fourth embodiment of a measuring device 
for carrying oiit the method of the inven- 
tion. 
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Fig. 6 a fifth embodiment of a measuring device for 
carrying out the method of the invention, 
and 

Fig. 7 a sixth embodiment of a measuring device for 
carrying out the method of the invention. 

The method of the invention is described below using 
Figs. 1, 2 and 3. The object 1 to be measured is, for 
example, fastened to a measuring table and illumi- 
nated by a projector 3 forming part of a sensor ar- 
rangement 2 on a stand. A camera 4 also forming part 
of the sensor arrangement 2 takes the picture of the 
illuminated object 1 or object area. In the practical 
example, the recording device designed as a camera is 
a CCD camera on a stand. 

In Fig. 2 the projector 3 and the camera 4 are inde- 
pendent of each other, while in Fig. 3 projector 3 
and camera 4 are rigidly connected to each other on a 
stand, which leads to simplification of recording the 
measurements. Figs. 2 and 3 show the sensor arrange- 
ment 2 in two different positions, position 1 being 
shown with unbroken lines and position 2 with broken 
lines . 

The projector 3 projects, onto the object 1 or object 
area to be measured, light patterns which are de- 
signed as line grids and/or Gray code sequences. The 
camera 4 records on each of its receiver elements the 
intensity of the strip images reproduced on the ob- 
ject 1, as measurement values. Then the grid and/or 
the Gray code sequence is turned through 90° and 
again projected onto the object 1, the axis of rota- 
tion lying parallel to the optical axis of the pro- 
jection system. 
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Furthermore, there is provided a so-called calibrat- 
ing camera 5 which is also designed as a CCD camera 
and takes pictures of the object 1 or an object area. 
5 Instead of the calibrating camera, photodetectors may 

be provided, and since only a limited number of meas- 
uring points have to be taken, as a minimum number 
only three photodetectors have to be used. 

10 In Fig. 1 are shown the basic relationships in the 

measuring cycle. From the sensor position 1 which is 
shown in unbroken lines in Figs. 2 and 3, as stated 
above, light patterns are projected with the projec- 
tor 3 onto the object 1 in such a way that two phase 

15 fields rotated through an angle (at the optimum 

through 90°) to each other are obtained, as a result 
of which each measuring point on the object 1 is sig- 
nalled with two phase values. The projector 3 pro- 
jects, for example, onto the object 1 to be measured 

20 or an object area, a series of two structure se- 

quences of out-of-phase grid lines and/or Gray code 
sequences, the two sequences having the rotation 
through 90° to each other. Like the measuring camera 
4, the calibrating camera 5 records the intensity 

2 5 measurement values of the individual images of the 

sequences on each of its receiver elements. 



With known algorithms of strip image evaluation, from 
the images recorded with the measuring camera 4 and 

30 the calibrating camera 5 can be calculated phase 

measurement values which correspond to the coordi- 
nates in the grid plane of the projection system, at 
each camera pixel point both for the measuring camera 
4 and for the calibrating camera 5, wherein, due to 

35 the selected type of strip projection, e.g. rotated 

grid sequences at the point of the object 1 under ob- 
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servation, up to two phase measurement values per di- 
rection of illumination are obtained. 

In the next step the sensor arrangement 2 is moved 
5 from the sensor position 1 into any other position, 

this position 2 being shown by the broken lines in 
Fig. 2 and Fig. 3. Here the station of the calibrat- 
ing camera 5 relative to the object 1 remains un- 
changed. From this new position of the sensor ar- 

10 rangement 2, further structure sequences or grid 

lines are projected by the projector 3 in the same 
way as stated above onto the object 1 or object area, 
wherein again both the measuring camera 4 and the 
calibrating camera 5 simultaneously take the picture 

15 sequences or parts of them. 

This process of switching the sensor arrangement 2 to 
other positions can be repeated until each area of 
the object to be measured has been detected at least 
once by the sensor arrangement 2, i.e. illuminated by 
the projector 3 and at the same time viewed with the 
measuring camera 4. In the process, the recording of 
measurements can be performed at the different posi- 
tions successively, and then the necessary calcula- 
tions can be made. However, calculations can also be 
made after each new position. 

The sensor arrangement 2 and the calibrating camera 5 
are connected to an evaluating device in which the 
30 necessary mathematical calculations are made for de- 

termining the three-dimensional coordinates of the 
object- Also, for evaluation of the phase information 
it is necessary for the geometry parameters of the 
projection system to be known. The spatial position 
35 of the individual projector positions is fixed by six 

external orientation parameters (three coordinates of 
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the centres of projection, three Eulerian angles of 
rotation about the coordinate axes rotated as well) . 
To calculate these geometry variables, a system of 
equations is set up with functional models of pro- 
grammetry. The four phase values which are measured 
or calculated from the measured values are used as 
input variables. Also the spatial position of the 
measuring camera 4, which is also determined by ori- 
entation parameters, is obtained through the evaluat- 
ing device. Here, the knowledge of the projection de- 
vice as well as two phase measurement values are 
used. 

The evaluation process will be explained below using 
Fig. 1. As already stated, the main condition of data 
evaluation is that the projector 3 signals the areas 
of the object 1 seen by the stationary calibrating 
camera 5 from at least two positions. Furthermore, 
care must be taken that the intensity measurement 
values for all lighting positions are always recorded 
by the calibrating camera 5 at the same object point 
Ok on the surface of the object 1, i.e. relative 
movement between the object 1 and the calibrating 
camera 5 must not take place. In Fig. 1 this condi- 
tion is shown with the aid of the beams 6, 7, 8 which 
all impinge on the object point Ok- The measuring 
camera 4 moved with it, on the other hand, detects 
different areas Oi for each position 1(1 = l...n) . For 
simplification, of these object areas only one point 
Oi or O2 is always shown in the schematic view of 
Fig. 1, 

Evaluation of the measuring operation for determining 
the 3-D coordinates of the surface of the object 1 is 
a three-stage process. Here basically only all pro- 
jector positions are calculated, i.e. calibrated. 
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then all measuring camera positions and finally the 
3-D coordinates of the coordinates of the object de- 
tected from the positions of the sensor arrangement 
2- This takes place, as stated above, after the com- 
plete recording of measurements or between the re- 
cording of measurements at different positions. 

First step of evaluation: 

In a first step of evaluation, using the projector 
image coordinates (^^i,k/ n^i^k) recorded on the pic- 
ture elements {±\, j\) with the stationary calibrat- 
ing camera 5, i.e. phase measurement values which de- 
scribe the object point .0^, the external and internal 
orientation parameters for all projector positions 1 
are calculated by known methods of beam block equali- 
sation. The main condition of this is, as stated 
above, that at least four phase images which have 
been generated during projection from at least two 
different projector positions 1 are available for the 
stationary camera. For each projector position, 
therefore, the six external orientation parameters 
which serve as the geometry parameters of the projec- 
tors and describe the spatial position of the indi- 
vidual projector positions are calculated. This cal- 
culation can be understood as calibration. The cali- 
brating camera as an overview camera thus serves to 
record the homologous points or their pixels as "vir- 
tual homologous points".. 

Up to now, only one calibrating camera 5 has been 
mentioned. However, there may be provided a plurality 
of calibrating cameras 5 which are in a fixed rela- 
tionship to the object point, the number of cameras 
being denoted as m (m ^ 1) . These calibrating cameras 
5 are arranged in different viewing positions, and so 



13 

a larger object area can be detected, i.e. more com- 
plicated objects can be measured. 

Second step of evaluation: 

5 

In position 1 of the sensor arrangement 2, the camera 
4 with its matrix of receiver elements photographs 
the object 1, wherein here one point Oi is picked out 
of the area which is shown by the beams 9 and 10. At 

10 the pixel point (^^i, t]\) of the camera 4 are ob- 

tained two items of measuring information (^^i,i, 
r|^i,i) in the form of phase measurement values which 
correspond to the projector image coordinates in the 
grid plane. From these items of information and the 

15 orientation parameters of the projector 3 at position 

1 calculated in the first step of evaluation, the 
orientation parameters of the measuring camera 4 at 
position 1 are calculated by free beam equalisation, 
i.e. the measuring camera is calibrated with respect 

20 to its position. In the process not just one point Oi 

from the object area detected is used, but up to (i x 
j) points can be used, where i x j is the pixel num- 
ber of the measuring camera 4 in row and column di- 
rections . 

25 

Afterwards the necessary orientation parameters which 
describe the sensor arrangement 2 at position 1 are 
available, wherein with the first step of evaluation 
the projector 3 and with the second step of evalua- 
30 tion the camera 4 are calibrated at this position . 

Third step of evaluation: 



35 



Using the orientation parameters of the projector 3 
and camera 4 at position 1, by conventional triangu- 
lation from the phase images are calculated the 
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three-dimensional coordinates of point Oi and hence 
of all object points (i x j) visible to the camera 4 
in this position 1 or view 1. As the object point was 
signalled by two grid sequences and so two phase im- 
5 ages are available, but only one phase measurement 

value is necessary, basically there is the option of 
calculating every point twice and settling it by av- 
eraging, so that the accuracy of measuring coordi- 
nates is increased. 

10 

Steps two and three of evaluation are performed for 
position 2 of the sensor arrangement 2 and then step- 
wise for all subsequent positions. ,As a result, a 3-D 
point cloud of the object is obtained with the meas- 
15 uring camera 4 from 1 viewing position and, if there 

is a plurality of calibrating cameras 5, additionally 
from m viewing positions. 

In Figs. 4 to 7 are shown other options for a device 
20 for carrying out the method. In Fig. 4 a plurality of 

calibrating cameras 5 are attached to a frame 11 be- 
low which is arranged the object 1 and which consists 
of two portal-like, parallel girder elements 12 and a 
cross beam 13 connecting the girder elements 12. On 
25 the cross beam 13, the sensor arrangement 2 is at- 

tached to a rotating part 14 rotatable in all spatial 
directions, wherein the camera and/or the projector 3 
too can be varied in their angle to the vertical. 

30 In Fig. 5 the object 1 is arranged on a measuring ta- 

ble 15 and a calibrating camera 5 is fixed to a frame 
16 above the object. The sensor arrangement 2 is 
again attached to a rotating unit 17 in the axis of 
rotation of which is located the measuring table 16. 

35 The sensor arrangement thus turns about the object 1 

at an angle to the vertical. 
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In Fig. 6 the measuring table is designed in the form 
of a turntable 18 which is rotatable relative to a 
base 19. The calibrating camera 5 is attached to the 
5 turntable by an arm 20 so as to stand over the object 

1. Measuring camera 4 and projector 3 of the sensor 
arrangement 2 are attached to a beam 21 which is rig- 
idly connected to the base 19. 

10 Finally, in Fig. 7 is shown a further device for car- 

rying out the method. Here, the sensor arrangement 2 
is attached to a circular guide track 22 rotatable 
about the object 1, in such a way that it can move in 
a semicircle and be positioned freely on the guide 

15 track, the guide track 22 being freely rotatable 

about the object 1 through an angle of up to 360°. A 
plurality of calibrating cameras 5 are attached to a 
frame 23 surrounding the guide track. Thus both the 
upper and the lower side of the object 1 can be de- 

20 tected. 



